Critical appraisal of machine learning prognostic models for acute pancreatitis: protocol for a systematic review.

Acute pancreatitis (AP) is an acute inflammatory disorder that is common, costly, and is increasing in incidence worldwide with over 300,000 hospitalizations occurring yearly in the United States alone. As its course and outcomes vary widely, a critical knowledge gap in the field has been a lack of accurate prognostic tools to forecast AP patients' outcomes. Despite several published studies in the last three decades, the predictive performance of published prognostic models has been found to be suboptimal. Recently, non-regression machine learning models (ML) have garnered intense interest in medicine for their potential for better predictive performance. Each year, an increasing number of AP models are being published. However, their methodologic quality relating to transparent reporting and risk of bias in study design has never been systematically appraised. Therefore, through collaboration between a group of clinicians and data scientists with appropriate content expertise, we will perform a systematic review of papers published between January 2021 and December 2023 containing artificial intelligence prognostic models in AP. To systematically assess these studies, the authors will leverage the CHARMS checklist, PROBAST tool for risk of bias assessment, and the most current version of the TRIPOD-AI. (Research Registry ( http://www.reviewregistry1727 .).

The Impact of Health Care Algorithms on Racial and Ethnic Disparities : A Systematic Review.

BACKGROUND: There is increasing concern for the potential impact of health care algorithms on racial and ethnic disparities. PURPOSE: To examine the evidence on how health care algorithms and associated mitigation strategies affect racial and ethnic disparities. DATA SOURCES: Several databases were searched for relevant studies published from 1 January 2011 to 30 September 2023. STUDY SELECTION: Using predefined criteria and dual review, studies were screened and selected to determine: 1) the effect of algorithms on racial and ethnic disparities in health and health care outcomes and 2) the effect of strategies or approaches to mitigate racial and ethnic bias in the development, validation, dissemination, and implementation of algorithms. DATA EXTRACTION: Outcomes of interest (that is, access to health care, quality of care, and health outcomes) were extracted with risk-of-bias assessment using the ROBINS-I (Risk Of Bias In Non-randomised Studies - of Interventions) tool and adapted CARE-CPM (Critical Appraisal for Racial and Ethnic Equity in Clinical Prediction Models) equity extension. DATA SYNTHESIS: Sixty-three studies (51 modeling, 4 retrospective, 2 prospective, 5 prepost studies, and 1 randomized controlled trial) were included. Heterogenous evidence on algorithms was found to: a) reduce disparities (for example, the revised kidney allocation system), b) perpetuate or exacerbate disparities (for example, severity-of-illness scores applied to critical care resource allocation), and/or c) have no statistically significant effect on select outcomes (for example, the HEART Pathway [history, electrocardiogram, age, risk factors, and troponin]). To mitigate disparities, 7 strategies were identified: removing an input variable, replacing a variable, adding race, adding a non-race-based variable, changing the racial and ethnic composition of the population used in model development, creating separate thresholds for subpopulations, and modifying algorithmic analytic techniques. LIMITATION: Results are mostly based on modeling studies and may be highly context-specific. CONCLUSION: Algorithms can mitigate, perpetuate, and exacerbate racial and ethnic disparities, regardless of the explicit use of race and ethnicity, but evidence is heterogeneous. Intentionality and implementation of the algorithm can impact the effect on disparities, and there may be tradeoffs in outcomes. PRIMARY FUNDING SOURCE: Agency for Healthcare Quality and Research.

Critical Appraisal for Racial and Ethnic Equity in Clinical Prediction Models Extension: Development of a Critical Appraisal Tool Extension to Assess Racial and Ethnic Equity-Related Risk of Bias for Clinical Prediction Models.

Introduction: Despite mounting evidence that the inclusion of race and ethnicity in clinical prediction models may contribute to health disparities, existing critical appraisal tools do not directly address such equity considerations. Objective: This study developed a critical appraisal tool extension to assess algorithmic bias in clinical prediction models. Methods: A modified e-Delphi approach was utilized to develop and obtain expert consensus on a set of racial and ethnic equity-based signaling questions for appraisal of risk of bias in clinical prediction models. Through a series of virtual meetings, initial pilot application, and an online survey, individuals with expertise in clinical prediction model development, systematic review methodology, and health equity developed and refined this tool. Results: Consensus was reached for ten equity-based signaling questions, which led to the development of the Critical Appraisal for Racial and Ethnic Equity in Clinical Prediction Models (CARE-CPM) extension. This extension is intended for use along with existing critical appraisal tools for clinical prediction models. Conclusion: CARE-CPM provides a valuable risk-of-bias assessment tool extension for clinical prediction models to identify potential algorithmic bias and health equity concerns. Further research is needed to test usability, interrater reliability, and application to decision-makers.

Clostridioides difficile dynamic electronic order panel, an effective automated intervention to reduce inappropriate inpatient ordering.

BACKGROUND: Ordering Clostridioides difficile diagnostics without appropriate clinical indications can result in inappropriate antibiotic prescribing and misdiagnosis of hospital onset C. difficile infection. Manual processes such as provider review of order appropriateness may detract from other infection control or antibiotic stewardship activities. METHODS: We developed an evidence-based clinical algorithm that defined appropriateness criteria for testing for C. difficile infection. We then implemented an electronic medical record-based order-entry tool that utilized discrete branches within the clinical algorithm including history of prior C. difficile test results, laxative or stool-softener administration, and documentation of unformed bowel movements. Testing guidance was then dynamically displayed with supporting patient data. We compared the rate of completed C. difficile tests after implementation of this intervention at 5 hospitals to a historic baseline in which a best-practice advisory was used. RESULTS: Using mixed-effects Poisson regression, we found that the intervention was associated with a reduction in the incidence rate of both C. difficile ordering (incidence rate ratio [IRR], 0.74; 95% confidence interval [CI], 0.63-0.88; P = .001) and C. difficile-positive tests (IRR, 0.83; 95% CI, 0.76-0.91; P < .001). On segmented regression analysis, we identified a sustained reduction in orders over time among academic hospitals and a new reduction in orders over time among community hospitals. CONCLUSIONS: An evidence-based dynamic order panel, integrated within the electronic medical record, was associated with a reduction in both C. difficile ordering and positive tests in comparison to a best practice advisory, although the impact varied between academic and community facilities.

Workplace Violence in Healthcare Settings: A Scoping Review of Guidelines and Systematic Reviews.

Workplace violence in healthcare settings is alarmingly common and represents significant financial and human cost. The aim of this scoping review was to identify and summarize evidence on strategies to prevent and/or manage workplace violence in healthcare settings. Searches were limited to evidence-based clinical practice guidelines and systematic reviews published between 2015 and 2021. Multiple databases were searched and screened. Quality of the included guidelines and reviews was also assessed. Three guidelines and 33 systematic reviews were included. Both the Occupational Safety and Health Administration 2015 and Registered Nurses' Association of Ontario 2019 guidelines provided useful recommendations for building a comprehensive prevention program. Evidence-based risk assessment, prevention and management, and education and training are all central components. Regular reassessment and adjustment is required. Included reviews (n = 33) were grouped into five main categories: violence toward nurses (n = 10); violence toward healthcare workers in general (n = 8); violence in the emergency department (n = 5); violence related to mental health (n = 5); and measurement related to workplace violence (n = 5). Multicomponent interventions were often more effective than those applied in isolation. We found consistent support for certain strategies including education and training, post-incident debriefing, multidisciplinary rapid response teams, and environmental modifications; however, the strength of evidence and certainty of conclusions were limited across reviews. This scoping review found that strong leadership that cultivates and enforces a culture of inclusivity, support, and respect is a prerequisite for a successful workplace violence prevention program. Rigorous comparative effectiveness research testing interventions are needed.

Interventions for malnutrition in hospitalized adults: A systematic review and meta-analysis.

BACKGROUND: Malnutrition is associated with poor outcomes in hospitalized adults. We aimed to assess the effectiveness of hospital-initiated interventions for patients with malnutrition. METHODS: Data sources included MEDLINE, Embase, Cochrane Library from January 1, 2000 to June 3, 2021. We included randomized controlled trials (RCTs) assessing interventions for hospitalized adults diagnosed or identified as at-risk for malnutrition using malnutrition screening and diagnostic assessment tools. Individual reviewers extracted study data and performed quality checks for accuracy. Meta-analysis was conducted using a random-effects model with variance correction. We assessed the overall strength of evidence at the outcome level. The risk of bias was assessed using the Cochrane Risk of Bias 2.0 tool. RESULTS: We found 11 RCTs that assessed two types of interventions: specialized nutrition care (8 RCTs) and increased protein provision (3 RCTs). The pooled findings of 11 RCTs found moderate strength of evidence that specialized nutrition care and increased protein provision reduced mortality by 21% (relative risk [RR]: 0.79, 95% confidence interval [CI]: 0.63-0.98; absolute risk reduction [ARR]: -0.02, 95% CI: -0.03 to -0.00). Pooled estimates indicated a nonsignificant decrease of 0.18 days in the length of stay (9 RCTs) and a 10% reduction in readmissions (7 RCTs). No eligible RCTs assessed parenteral or enteral nutrition. CONCLUSION: Certain malnutrition-focused hospital-initiated interventions (e.g., specialized nutrition care and increased protein provision) reduce mortality and may improve the quality of life among patients at risk for or diagnosed with malnutrition. Future trials are needed to assess the effectiveness of parenteral and enteral nutrition.

Consumer Devices for Patient-Generated Health Data Using Blood Pressure Monitors for Managing Hypertension: Systematic Review.

BACKGROUND: In the era of digital health information technology, there has been a proliferation of devices that collect patient-generated health data (PGHD), including consumer blood pressure (BP) monitors. Despite their widespread use, it remains unclear whether such devices can improve health outcomes. OBJECTIVE: We performed a systematic review of the literature on consumer BP monitors that collect PGHD for managing hypertension to summarize their clinical impact on health and surrogate outcomes. We focused particularly on studies designed to measure the specific effect of using a BP monitor independent of cointerventions. We have also summarized the process and consumer experience outcomes. METHODS: An information specialist searched PubMed, MEDLINE, and Embase for controlled studies on consumer BP monitors published up to May 12, 2020. We assessed the risk of bias using an adapted 9-item appraisal tool and performed a narrative synthesis of the results. RESULTS: We identified 41 different types of BP monitors used in 49 studies included for review. Device engineers judged that 38 (92%) of those devices were similar to the currently available consumer BP monitors. The median sample size was 222 (IQR 101-416) participants, and the median length of follow-up was 6 (IQR 3-12) months. Of the included studies, 18 (36%) were designed to isolate the clinical effects of BP monitors; 6 of the 18 (33%) studies evaluated health outcomes (eg, mortality, hospitalizations, and quality of life), and data on those outcomes were unclear. The lack of clarity was due to low event rates, short follow-up duration, and risk of bias. All 18 studies that isolated the effect of BP monitors measured both systolic and diastolic BP and generally demonstrated a decrease of 2 to 4 mm Hg in systolic BP and 1 to 3 mm Hg in diastolic BP compared with non-BP monitor groups. Adherence to using consumer BP monitors ranged from 38% to 89%, and ease of use and satisfaction ratings were generally high. Adverse events were infrequent, but there were a few technical problems with devices (eg, incorrect device alerts). CONCLUSIONS: Overall, BP monitors offer small benefits in terms of BP reduction; however, the health impact of these devices continues to remain unclear. Future studies are needed to examine the effectiveness of BP monitors that transmit data to health care providers. Additional data from implementation studies may help determine which components are critical for sustained BP improvement, which in turn may improve prescription decisions by clinicians and coverage decisions by policy makers.

Interventions to Reduce Hospital Length of Stay in High-risk Populations: A Systematic Review.

Importance: Many strategies to reduce hospital length of stay (LOS) have been implemented, but few studies have evaluated hospital-led interventions focused on high-risk populations. The Agency for Healthcare Research and Quality (AHRQ) Learning Health System panel commissioned this study to further evaluate system-level interventions for LOS reduction. Objective: To identify and synthesize evidence regarding potential systems-level strategies to reduce LOS for patients at high risk for prolonged LOS. Evidence Review: Multiple databases, including MEDLINE and Embase, were searched for English-language systematic reviews from January 1, 2010, through September 30, 2020, with updated searches through January 19, 2021. The scope of the protocol was determined with input from AHRQ Key Informants. Systematic reviews were included if they reported on hospital-led interventions intended to decrease LOS for high-risk populations, defined as those with high-risk medical conditions or socioeconomically vulnerable populations (eg, patients with high levels of socioeconomic risk, who are medically uninsured or underinsured, with limited English proficiency, or who are hospitalized at a safety-net, tertiary, or quaternary care institution). Exclusion criteria included interventions that were conducted outside of the hospital setting, including community health programs. Data extraction was conducted independently, with extraction of strength of evidence (SOE) ratings provided by systematic reviews; if unavailable, SOE was assessed using the AHRQ Evidence-Based Practice Center methods guide. Findings: Our searches yielded 4432 potential studies. We included 19 systematic reviews reported in 20 articles. The reviews described 8 strategies for reducing LOS in high-risk populations: discharge planning, geriatric assessment, medication management, clinical pathways, interdisciplinary or multidisciplinary care, case management, hospitalist services, and telehealth. Interventions were most frequently designed for older patients, often those who were frail (9 studies), or patients with heart failure. There were notable evidence gaps, as there were no systematic reviews studying interventions for patients with socioeconomic risk. For patients with medically complex conditions, discharge planning, medication management, and interdisciplinary care teams were associated with inconsistent outcomes (LOS, readmissions, mortality) across populations. For patients with heart failure, clinical pathways and case management were associated with reduced length of stay (clinical pathways: mean difference reduction, 1.89 [95% CI, 1.33 to 2.44] days; case management: mean difference reduction, 1.28 [95% CI, 0.52 to 2.04] days). Conclusions and Relevance: This systematic review found inconsistent results across all high-risk populations on the effectiveness associated with interventions, such as discharge planning, that are often widely used by health systems. This systematic review highlights important evidence gaps, such as the lack of existing systematic reviews focused on patients with socioeconomic risk factors, and the need for further research.

Translating an evidence-based clinical pathway into shareable CDS: developing a systematic process using publicly available tools.

OBJECTIVE: To develop a process for translating semi-structured clinical decision support (CDS) into shareable, computer-readable CDS. MATERIALS AND METHODS: We developed a systematic and transparent process using publicly available tools (eGLIA, GEM Cutter, VSAC, and the CDS Authoring Tool) to translate an evidence-based clinical pathway (CP) into a Clinical Quality Language (CQL)-encoded CDS artifact. RESULTS: We produced a 4-phase process for translating a CP into a CQL-based CDS artifact. CP content was extracted using GEM into discrete clinical concepts, encoded using standard terminologies into value sets on VSAC, evaluated against workflows using a wireframe, and finally structured as a computer readable CDS artifact using CQL. This process included a quality control step and intermediate products to support transparency and reuse by other CDS developers. DISCUSSION: Translating a CP into a shareable, computer-readable CDS artifact was accomplished through a systematic process. Our process identified areas of ambiguity and gaps in the CP, which generated improvements in the CP. Collaboration with clinical subject experts and the CP development team was essential for translation. Publicly available tools were sufficient to support most translation steps, but expression of certain complex concepts required manual encoding. CONCLUSION: Standardized development of CDS from a CP is feasible using a systematic 4-phase process. CPs represent a potential reservoir for developers of evidence-based CDS. Aspects of CP development simplified portions of the CDS translation process. Publicly available tools can facilitate CDS development; however, enhanced tool features are needed to model complex CDS statements.

A Multimodal Intervention Using Nonopioid Analgesics Is Associated With Reduced Intravenous Opioid Exposure Among Hospitalized Patients With Inflammatory Bowel Diseases.

INTRODUCTION: Opioid use in patients with inflammatory bowel disease (IBD) is associated with increased mortality. Previous interventions targeting reduced intravenous opioid (IVOPI) exposure for all patients admitted to a general medical unit have decreased total opioid use without compromising pain control. We therefore performed a prospective evaluation of a multimodal intervention encouraging the use of nonopioid alternatives to reduce IVOPI exposure among patients with IBD hospitalized at our institution. METHODS: This was a prospective evaluation of a multimodal intervention to reduce IVOPI use among patients with IBD aged ≥18 years admitted to a general medical unit at a large urban academic medical center from January 1, 2019, to June 30, 2019. Intravenous and total (all routes) opioid exposures were measured as proportions and intravenous morphine milligram equivalents/patient day and compared with preintervention (January 1, 2018, to December 31, 2018) data. Hospital length of stay (LOS), 30-day readmission rates (RRs), and pain scores (1-10 scale) were also assessed. RESULTS: Our study involved 345 patients with IBD with similar baseline characteristics in preintervention (n = 241) and intervention (n = 104) periods. Between study periods, we observed a significant reduction in the proportion of patients receiving IVOPIs (43.6% vs 30.8%, P = 0.03) and total opioid dose exposure (15.6 vs 8.5 intravenous morphine mg equivalents/d, P = 0.02). We observed similar mean pain scores (3.9 vs 3.7, P = 0.55) and significantly reduced mean LOS (7.2 vs 5.3 days, P = 0.03) and 30-day RRs (21.6% vs 11.5%, P = 0.03). DISCUSSION: A multimodal intervention was associated with reduced opioid exposure, LOS, and 30-day RRs for hospitalized patients with IBD. Additional research is needed to determine long-term benefits of reduced opioid exposure in this population.

AHRQ EPC Series on Improving Translation of Evidence: Use of a Clinical Pathway for C. Difficile Treatment to Facilitate the Translation of Research Findings into Practice.

BACKGROUND: In 2018 the Agency for Healthcare Research and Quality (AHRQ) Evidence-based Practice Center (EPC) Program issued a call for strategies to disseminate AHRQ EPC systematic reviews. In this pilot, findings from the 2016 AHRQ EPC report on Clostridioides difficile infection were translated into a treatment pathway and disseminated via a cloud-based platform and electronic health record (EHR). METHODS: An existing 10-step framework was used for developing and disseminating evidence-based clinical pathways. The development of the EHR intervention was informed by the Five Rights model for clinical decision support and human-computer interaction design heuristics. The researchers used observations and time measurements to describe the impact of the EPC report on pathway development and examined provider adoption using counts of pathway views. RESULTS: Two main themes emerged: (1) discrepancies between the EPC report and existing guidelines prompted critical discussions about available treatments, and (2) lack of guideline and pathway syntheses in the EPC report necessitated a rapid literature review. Pathway development required 340 hours: 205 for the rapid literature review, 63 for pathway development and EHR intervention design, and 5 for technical implementation of the intervention. Pathways were viewed 1,069 times through the cloud-based platform and 47 times through a hyperlink embedded in key EHR ordering screens. CONCLUSION: Pathways can be an approach for disseminating AHRQ EPC report findings within health care systems; however, reports should include guideline and pathway syntheses to meet their full potential. Embedding hyperlinks to pathway content within the EHR may be a viable and low-effort solution for promoting awareness of evidence-based resources.

Impact of a Hospital Evidence-Based Practice Center (EPC) on Nursing Policy and Practice.

BACKGROUND: In 2006, our healthcare system created a hospital Evidence-based Practice Center (EPC) to support the local delivery of high-quality, safe and high value patient care. Since then, the importance of healthcare staff work life has also been highlighted, and together these four elements form the Quadruple Aim framework. Synergistic to this Aim, the Magnet® program promotes and recognizes organizational nursing excellence. OBJECTIVE: To examine the EPC's work to inform nursing policy and practice in support of the goals of the Quadruple Aim framework and Magnet® designation. METHODS: Methods used included the following: (1) descriptive analysis of the hospital EPC's database of rapid reviews; and (2) administration of a 40-item electronic questionnaire to nurses who requested an EPC review during fiscal years (FY) 2015 and 2016. RESULTS: Of 308 rapid reviews completed in the EPC's first 10 years, 59 (19%) addressed nursing topics. The proportion of reviews relevant to nursing increased from 5% (2/39) in the center's first 2 years to 44% (25/60) in FY 2015-2016. The majority of nursing reviews (39/59) examined processes of care. Of 23 nurses eligible to participate in the survey, 21 responded (91%). Nurses with administrative or managerial responsibilities requested 70% of reviews; clinical nurse specialists and bedside nurses requested 17% and 9%, respectively. Reviews were used to support clinical program development (48%), provide clinical guidance (33%), update nursing policies or procedures (24%) and develop training and curricula (24%). Nurses were satisfied with the hospital EPC reviews (mean; 4.7/5), and 95% indicated they were likely to request a future review. LINKING EVIDENCE TO ACTION: A dedicated hospital EPC in partnership with nursing offers a unique mechanism for promoting a culture of evidence-based practice. Nurses at all organizational levels use the services of a hospital EPC to inform nursing policy and practice and are highly satisfied with the process, supporting the Quadruple Aim and Magnet® designation.

Using a 10-step framework to support the implementation of an evidence-based clinical pathways programme.

BACKGROUND: Integration of evidence into practice is suboptimal. Clinical pathways, defined as multidisciplinary care plans, are a method for translating evidence into local settings and have been shown to improve the value of patient care. OBJECTIVE: To describe the development of a clinical pathways programme across a large academic healthcare system. METHODS: We use a 10-step framework (grounded in the Knowledge-to-Action framework and ADAPTE Collaboration methodology for guideline adaptation) to support pathway development and dissemination, including facilitating clinical owner and stakeholder engagement, developing pathway prototypes based on rapid reviews of the existing literature, developing tools for dissemination and impact assessment. We use a cloud-based technology platform (Dorsata, Washington, DC) to assist with development and dissemination across our geographically distributed care settings and providers. Content is viewable through desktop and mobile applications. We measured programme adoption and penetration by examining number of pathways developed as well as mobile application use and pathway views. RESULTS: From 1 February 2016 to 30 April 2018, a total of 202 pathways were disseminated. The three most common clinical domains represented were oncology (46.5%, n=94), pulmonary/critical care (8.9%, n=18) and cardiovascular medicine (7.4%, n=15). Users opting to register for a personal account totalled 1279; the three largest groups were physicians (45.1%, n=504), advanced practice providers (19.5%, n=245) and nurses (19.1%, n=240). Pathway views reached an average of 2150 monthly views during the last 3 months of the period. The majority of pathways reference at least one evidence-based source (93.6%, n=180). CONCLUSIONS: A healthcare system can successfully use a framework and technology platform to support the development and dissemination of pathways across a multisite institution.

Predictors of Short Intensive Care Unit Stay for Patients with Diabetic Ketoacidosis Using a Novel Emergency Department-Based Resuscitation and Critical Care Unit.

BACKGROUND: The resuscitation and critical care unit is a novel emergency department-intensive care unit designed to provide early critical care to emergency department patients for ≤24 h. OBJECTIVES: This study sought to identify clinical variables associated with short intensive care unit (ICU) stays in patients with diabetic ketoacidosis (DKA), who commonly require ICU-level care. METHODS: We conducted a retrospective, single-center, cross-sectional study of DKA patients ≥18 years of age who presented to an academic, urban hospital emergency department over 16 months. Patient demographics and clinical variables extracted from medical records were compared between prolonged ICU stay patients of ≥24 h versus short ICU stay patients (SSPs) of <24 h. ICU care was defined as treatment in the resuscitation and critical care unit or inpatient ICU. RESULTS: One hundred sixty-eight emergency department visits with a primary diagnosis of DKA were analyzed. There were 53 prolonged ICU stay patients, 58 SSPs, and 57 patients required no ICU time. SSPs had significantly higher initial serum bicarbonate (13.0 vs. 9.0 mEq/L, p = 0.01) and shorter anion gap closure time (9.8 vs. 14.4 hours, p = 0.003). Medication nonadherence was a significantly more frequent precipitant in SSPs (67.2% vs. 47.2%, p = 0.03). Initial anion gap, glucose, beta-hydroxybutyrate, and severity of illness scores were not significantly different between groups. After multivariate logistic regression adjusting for variables significant from univariate analysis, higher initial bicarbonate (p = 0.04) and medication nonadherence (p = 0.03) remained significantly associated with SSPs. CONCLUSIONS: Patients with DKA with short ICU stays have higher initial bicarbonate levels and are more likely to have medication nonadherence than patients requiring prolonged critical care. These variables may identify patients with DKA who are best treated in an emergency department-intensive care unit to potentially reduce inpatient ICU use.